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BEHAVIORAL  ASPECTS  OF  £OST-|ENEFIT  ANALYSIS . 


BARUCH  FISCHHOFF 


SUMMARY 


Over  recent  years,  the  decision-making  habits  of  military  officers,  govern¬ 
ment  bureaucrats,  business  executives  and  labor  leaders  have  gradually 
changed.  Increasingly,  when  time  permits,  they  are  likely  to  order  or 
conduct  personally  a  formal  analysis  of  their  decision  problem.  Most 
analyses  have  what  might  be  called  a  "cost-benefit"  logic.  For  each 
possible  action,  one  first  enumerates  all  possible  consequences,  and  then 
assesses  how  likely  they  are  to  happen  and  how  good  and  bad  they  would  be. 
Straight-forward  computations  produce  the  expected  cost  and  expected 
benefit  of  the  action.  Usually,  a  preponderance  of  benefits  over  costs 
Is  a  precondition  for  action.  Any  sound  analysis  will  consider  all 
possible  actions  (Including  Inaction)  In  this  light. 

As  the  adoption  of  such  procedures  has  grown,  so  has  the  scientific 
understanding  of  their  validity.  This  paper  reviews  that  literature  with 
a  focus  on  decision  making  In  the  public  realm.  Three  generic  sets  of 
limits  are  described:  (a)  the  unavailability  of  vital  Inputs  (i.e.,  values 
for  important  costs  and  benefits),  (b)  the  absence  of  procedures  for 
assessing  the  validity  of  an  analysis,  and  (c)  the  necessary  Incompleteness 
of  any  formal  model  of  a  decision  situation.  These  limits  have  not 
always  been  fully  appreciated  In  conducting  analyses  In  the  past. 

Despite  these  deficiencies,  however,  cost-benefit  analyses  may  have  an 
Important  role  to  play  In  decision  making,  particularly  when  one  considers 
the  weaknesses  of  alternative  approaches.  Realization  of  this  potential 
will  require  sensitivity  to  Its  strength  and  limits  and  to  the  complexity 
of  the  social  and  political  situations  In  which  it  Is  applied. 
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1.  OVERVIEW 


Cost-benefit  analysis  asks  whether  the  expected  benefits  from  a  proposed 
activity  outweigh  Its  expected  costs.  Prominent  variants  are  cost- 
effective  analysis  (comparing  the  relative  benefits  of  alternative 
actions),  risk-benefit  analysis  (In  which  the  major  costs  are  risks  to 
life,  limb  or  property),  and  decision  analysis  (In  which  the  need  to 
reduce  all  consequences  to  monetary  units  Is  avoided  and  the  role  of 
uncertainty  Is  made  more  explicit).* 

The  expected  cost  of  a  project  Is  determined  by  enumerating  all  aversive 
consequences  that  might  arise  from  Its  Implementation  (e.g..  Increased 
occupational  hazard),  assessing  the  probability  that  each  will  occur,  and 
estimating  the  cost  or  loss  to  society  should  each  occur.  Next,  the 
expected  loss  from  each  possible  consequence  Is  calculated  by  multiplying 
the  amount  of  the  loss  by  the  probability  that  It  will  be  Incurred.  The 
expected  loss  of  the  entire  project  Is  computed  by  sunning  the  expected 
losses  associated  with  the  various  possible  consequences.  An  analogous 
procedure  produces  an  estimate  of  the  expected  benefits. 

Although  based  on  an  appealing  premise  and  supported  by  a  sophisticated 
methodology,  these  procedures  have  a  number  of  characteristic  limits  on 
their  usefulness  as  management  tools.  These  limits  arise  when  the  mathe¬ 
matical  formalisms  c  >nt  the  fallible  Individuals  who  mus  conduct, 
accept,  or  Implement  tr.em.  Be  they  technical  experts,  lay  Interveners  or 

^Excellent  expositions  of  these  procedures  may  be  found  In  a  number  of 
sources  Including  R.l.  Keeney  &  H.  Ralffa,  Decisions  with  Multiple 
Objectives  (New  York:  Wiley,  1976);  R.  Layard  (Ed.).  Cost  Benefit  Analy¬ 
sis  (Harmondsworth:  Penguin,  1974);  E.  Stokey  &  R.  Zeckhauser,  A  Primer 
for  Policy  Analysis  (New  York:  Norton,  1978);  H.  Ralffa,  Decision  Analy¬ 
sis  (Reading:  Addison  Wesley,  1968). 


government  regulators,  these  Individuals  all  have,  to  some  extent,  limited 
capacity  to  process  technical  Information,  restricted  resources  to  devote 
to  the  project  at  hand.  Irrational  apprehensions  about  Its  consequences. 
Intransigent  prejudices  about  the  facts  of  the  matter,  ulterior  motives, 
and  Incoherent  and  unstable  values  on  critical  Issues.  Deliberately  or 
Inadvertently,  these  human  properties  tend  to  foil  the  best  laid  plans  of 
cost-benefit  analysts  or  the  purveyors  of  almost  any  other  scheme  for 
managing  technologies  In  society. 

In  considering  the  viability  of  any  approach  to  management.  It  seems  to  be 
useful  to  have  In  mind  a  set  of  desiderata  such  as  that  In  Table  1.  Although 
this  list  Is  by  no  means  definitive.  It  suggests  the  criteria  a  proposed 
approach  must  meet  In  order  to  merit  wholehearted  adoption.  No  proposal 
achieves  perfect  marks  In  each  category.  Analyzing  the  strengths  and 
weaknesses  of  each  proposal  can  show  where  It  needs  Improvement  and  for 
which  problems  It  might  be  most  suitable. 

Rather  than  consider  cost-benefit  analysis  from  each  of  these  perspectives, 
some  of  which  are  covered  by  other  addresses  In  this  symposium,  I  will 
discuss  the  effects  of  three  sets  of  limits  on  members  of  this  family  of 
analytical  procedures.  One  set  of  limits  Is  Imposed  by  the  unavailability 
of  necessary  Inputs  to  the  analysis.  Without  those  Inputs,  the  Implement- 
ability  of  cost-benefit  analysis  must  be  seriously  questioned.  A  second 
set  of  limits  comes  froa  the  Inability  of  the  analysts  to  assess  the 
validity  of  their  work  and  Incorporate  that  assessment  Into  their  guides 
to  action.  The  absence  of  such  appraisals  creates  problems  for  the  logical 
soundness  of  such  analyses  and  their  ability  to  protect  the  public  from 
unanticipated  side  effects.  The  third  set  of  limits  comes  from  the 
failure  of  these  methods  to  address  critical  management  Issues.  These 
Include  the  acceptability  of  the  political  philosophy  underlying  the 
procedures  and  the  feasibility  of  Implementing  their  reconmendatlons. 
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TABLE  1 


CRITERIA  FOR  EVALUATING  APPROACHES 
TO  MANAGING  TECHNOLOGIES 

Logically  Sound 

Implemen  table 

Politically  Acceptable 

Respects  Institutional  Constraints 

Open  to  Evaluation 

Creates  no  Side  Effects 


Promotes  Effective  Long-Term  Management 


2.  AVAILABILITY  OF  INPUTS 


Performing  a  full-dress  analysis  assumes,  among  other  things,  that  (1) 
all  possible  events  and  all  significant  consequences  can  be  enumerated 
In  advance;  (2)  meaningful  probability,  cost  and  benefit  values  can  be 
produced  and  assigned  to  them;  and  (3)  the  often  disparate  costs  and 
benefits  can  somehow  be  compared  to  one  another. 

Unfortunately,  some  of  these  tasks  cannot  be  completed  at  all;  while  for 

others,  the  results  are  hardly  to  be  trusted.  Despite  the  enormous 

scientific  progress  of  the  last  decade  or  two,  we  still  do  not  know  all 

of  the  possible  physical,  biological  and  social  consequences  of  any 

large-scale  energy  project.  Where  we  know  what  the  consequences  are,  we 

often  do  not,  or  cannot,  know  their  likelihoods.  For  example,  although 

we  know  that  a  reactor  core  melt-down  Is  unlikely,  we  will  not  know  quite 

how  unlikely  until  we  accumulate  much  more  on-line  experience.  Even  then, 

we  will  be  able  to  utilize  that  knowledge  only  If  we  can  assume  that  the 

system  and  Its  surrounding  conditions  remain  the  same  (e.g.,  there  will 

be  no  changes  In  the  Incidence  of  terrorism  or  the  availability  of 

trained  personnel).  For  many  situations,  even  when  danger  Is  known  to 

be  present.  Its  extent  cannot  be  known.  Whenever  low-level  radiation 

or  exposure  to  toxic  substances  Is  Involved,  consequences  can  be  assessed 

only  by  somewhat  tenuous  extrapolation  from  the  consequences  of  high- 

2 

level  exposure  to  humans  or  low-level  exposure  to  animals. 

In  all  these  cases,  we  must  rely  upon  unaided  human  judgment  to  guide  or 
supplant  our  formal  methods.  Research  Into  the  psychological  processes 
Involved  In  producing  such  judgments  offers  reasons  for  pessimism.  A 
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Harriss,  R.,  Hohenemser,  C.  &  Kates,  R.  The  Burden  of  Technological 
Hazards,  Environment,  1978,  8. 


rather  robust  result  Is  that  people  have  a  great  deal  of  difficulty 
both  in  comprehending  Information  under  conditions  of  complexity  and 
uncertainty  and  In  making  valid  Inferences  from  such  information.  The 
fallibility  of  such  judgment  stems  In  part  from  the  counter-intuitive 
nature  of  many  probabilistic  processes,  in  part  from  the  lack  of  hands- 
on  experience  with  low-probability  and  high-consequence  events,  and  In 
part  from  the  mental  overload  created  by  many  problems. 

The  fallings  of  our  Intuitions  are  shown  In  persistent  tendencies  to 
neglect  various  kinds  of  normatlvely  Important  Information,  such  as 
population  base  rates  (Indicating  how  conmon  a  particular  event  Is), 
sample  size  (Indicating  how  reliable  evidence  Is)  and  predictive  validity. 
Other  kinds  of  Information  are  attended  to,  but  given  Inappropriate 
Interpretations.  People  tend  to  be  more  confident  making  predictions  on 
the  basis  of  redundant  Information  than  with  Independent  information 
(although  the  latter  has  greater  predictive  validity).  They  readily  find 
Interpretable  patterns  in  random  sequences,  and  they  assume  that  more 
Information  guarantees  better  performance  even  when  It  only  generates 
confusion.  When  asked  to  synthesize  Information  from  their  experience, 
people  tend  to  misjudge  the  risks  to  which  they  are  exposed.  They 
remember  themselves  to  have  been  more  foresightful --and  others  to  have 
been  less  foresightful— In  past  judgments  than  was  actually  the  case, 
and  they  sometimes  persevere  in  erroneous  beliefs  despite  mounting,  even 

4 

overwhelming,  contrary  evidence. 


3Slov1c,  P.,  Flschhoff,  B.  &  Lichtenstein,  S.  Behavioral  decision 
theory.  Annual  Review  of  Psychology,  1977,  28,  1-39. 

*Tversky,  A.  &  Kahneman,  D.  Judgment  under  uncertainty:  Heuristics  and 
biases.  Science,  1974,  185,  1124-1131;  Flschhoff,  B.  Hindsight  t  Fore¬ 
sight:  The  effect  of  ourcome  knowledge  on  judgment  under  uncertainty. 
Journal  ^Experimental  Psychology:  Human  Perception  and  Performance. 
1975,  U '  28E-299;  L.  Ross  lThe  Intuitive  psychologist  and  his  short- 
comlngs"  Ip  L.  Berkowltz  (Ed.),  Advances  In  Social  Psychology  (New 
York:  Arxjemlc  Press,  1977.) 
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To  the  best  of  our  understanding,  these  judgmental  biases  do  not  reflect 
folly,  but  the  occasional  (or  frequent)  inadequacies  of  people's  best 
attempts  to  muddle  through  difficult  problems.  Often  these  attempts 
reflect  the  use  of  judgmental  heuristics  or  rules  of  thumb  that  embody 
moderately  valid  appraisals  of  regularities  in  the  world  around  us.  In 
situations  allowing  a  trlal-and-error,  successive  correction  approach 
to  decision  problems,  these  heuristics  often  work  pretty  well.  However, 
where  we  (as  planners  or  consumers)  must  get  our  decisions  right  the 
first  time  or  suffer  severe  consequences,  the  limits  of  these  heuristics 
may  spell  real  problems. 

These  problematic  tendencies  are  typically  observed  In  situations  where 
people's  sole  motivation  would  seem  to  be  making  the  best,  most  "objec¬ 
tive"  judgment  possible.  Such  situations  are  designed  to  avoid  the 
additional  problems  engendered  by  the  coloring  of  judgments  by  wishful 
thinking,  self-serving  motives,  selective  attention,  cognitive  dissonance 
and  the  like.  Although  the  evidence  is  sketchy,  there  is  at  the  moment 
no  good  empirical  reason  to  believe  that  these  problems  are  appreciably 
reduced  when  the  judgment  in  question  carries  high  (personal  or  societal) 
stakes  or  when  the  judge  Is  a  substantive  expert  forced  to  go  beyond 
the  available  data  and  rely  on  intuition.^ 

Such  results  provide  strong  evidence  for  using  formal  methods  for  pro¬ 
ducing  and  combining  information  whenever  possible— and  for  treating 
the  results  of  such  analyses  with  considerable  caution  because  of 
their  Inevitable  judgmental  component. 


Slovic,  P.,  Fischhoff,  B.  &  Lichtenstein,  S.  op  clt.;  Fischhoff,  B. 
Cost  benefit  analysis  and  the  art  of  motorcycle  maintenance.  Policy 
Sciences.  1977,  8,  177-202. 
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Once  the  consequences  have  been  enumerated  and  their  likelihood  assessed, 
a  price  tag  (In  dollars  or  utiles)  must  be  placed  on  them.  When  It  comes 
to  tradeoffs  between  deaths  today  and  In  the  future,  between  sterility 
and  black  lung  disease  or  between  profits  and  lives,  both  the  exigencies 
of  our  political  processes  and  the  Indeterminate  nature  of  cost-benefit 
logic  force  us  to  ask  people  for  their  opinions. 

Such  questions  of  value  would  seem  to  be  the  last  redoubt  of  intuitive 
judgment.  Unfortunately,  however,  subtle  changes  In  how  questions  are 
posed  can  have  a  major  impact  on  the  opinions  elicited.  Worse  yet.  In 
situations  where  alternative  questioning  procedures  elicit  different 
preferences,  the  normative  theory  often  offers  no  guide  as  to  which  of 
the  different  judgments  Is  to  be  preferred.  When  people's  judgments 
show  this  sort  of  lability,  the  method  may  become  the  message,  leading 
to  decisions  not  in  the  decision  makers'  best  interest,  to  action  when 
caution  is  desirable  (or  the  opposite),  or  to  the  obfuscation  of  poorly 
articulated  views.® 

Many  of  t-hese  effects  have  been  known  since  the  antiquity  of  experi¬ 
mental  psychology  In  the  mid-1800's.  Early  psychologists  concerned  with 
the  relationship  between  sensations  and  judgments  about  them  found  that 
both  the  threshold  for  discerning  a  sensation  and  the  threshold  for  dis¬ 
criminating  between  two  sensations  depended  on  a  variety  of  subtle 
aspects  of  how  stimuli  were  presented  and  how  responses  were  elicited. 
Different  judgments  were  attached  to  the  same  stimuli  as  a  function  of 
whether  those  stimuli  were  presented  In  ascending  (Increasing  on  a 
physical  continuum)  or  descending  order,  whether  the  set  of  stimuli  was 


®F1schhoff,  B.,  Slovic,  P.  &  Lichtenstein,  S.  Knowing  what  you  want: 
Measuring  labile  values.  In  T.  Wallsten  (Ed.),  Cognitive  Processes 
In  Choice  and  Decision  Behavior.  (Hillsdale,  N.J7:  Erlbaum,  In  press.) 
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homogeneous  or  diverse,  whether  particular  regions  on  the  continuum  were 
densely  or  sparsely  represented,  whether  sequentially  presented  stimuli 
were  relatively  similar  or  disparate,  whether  values  near  the  threshold 
of  detection  were  Included  or  not,  and  whether  the  respondent  made  one 
or  many  judgnents.  Even  when  the  same  presentation  was  used,  different 
jud^nents  might  be  obtained  with  a  numerical  or  comparative  (ordinal) 
response  mode,  with  Implicit  Instructions  motivating  speed  versus  accuracy, 
with  a  bounded  or  unbounded  response  set,  with  small  or  large  numbers 
(subsequently  normalized),  or  with  verbal  or  numerical  labels.  The 
Instability  of  judgment  Is  heightened  by  the  fact  that  perception  Is 
Inherently  accompanied  by  some  random  error  and  by  Idiosyncratic  tendencies 
such  as  fatigue,  locking  In  on  stereotypic  ways  of  viewing  a  problem, 
second-guessing  the  ellcltor  (what  am  I  supposed  to  say?)  and  linking 
variables  that  should  be  Independent  (halo  effects). 

All  of  these  problems  emerge  when  people  are  questioned  about  their  values 
or  preferences.  The  ellcltor  must  decide  how  many  questions  to  ask  and 
how  to  word  them;  what  response  format  to  use  and  how  much  time  to  allot 
for  It.  The  preferences  expressed  will  reflect  In  part  the  respondent's 
true  beliefs,  In  part  the  method  used  to  uncover  them.  Indeed,  no  decision 
Is  so  clear  cut  In  Its  options,  events  and  attributes,  no  respondent  Is  so 
mechanical,  that  these  problems  can  be  avoided  entirely. 

Particular  kinds  of  lability  seem  to  emerge  when  people  are  asked  about 
value  Issues  of  the  sort  raised  by  proposed  energy  strategies.  For  such 
new  and  complex  Issues,  with  subtle  Interactions  and  gargantuan  effects, 
people  may  have  no  articulated  preferences.  In  some  fundamental  sense, 
their  values  are  Incoherent,  not  thought  through.  The  desires  they  express 
at  any  particular  time  are  those  tapped  by  the  particular  question  posed. 
That  question  may  evoke  a  central  concern  or  a  peripheral  one;  It  may  help 
clarify  the  respondent's  opinion  or  Irreversibly  shape  It;  It  may  even 
create  an  opinion  where  none  existed  before. 
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Listing  a  few  specific  effects  nay  Indicate  the  power  an  el  Id  tor  may 
deliberately,  or  Inadvertently,  wield  In  shaping  expressed  preferences. 

The  desirability  of  possible  outcomes  Is  often  evaluated  In  relation  to 
some  reference  point.  That  point  could  be  one's  current  (asset)  position, 
or  an  expected  level  of  wealth  (what  someone  with  my  talents  should  be 
worth  at  time  t),  or  that  possessed  by  another  person.  Shifts  In 
reference  point  are  fairly  easily  effected  and  can  lead  to  appreciable 
shifts  In  judged  desirability,  even  to  reversals  In  the  order  of  pre¬ 
ference.  Consider,  for  example,  how  one  might  think  about  the  same 
safety  program  conceptualized  In  terms  of  lives  saved  or  lives  lost, 
with  the  respective  reference  points  of  the  current  situation  or  an 
Ideal  one.  As  one  gets  closer  to  an  event  with  mixed  consequences,  the 
aversiveness  of  Its  negative  aspects  may  Increase  more  rapidly  than  the 
attractiveness  of  Its  positive  aspects,  making  It  appear,  on  the  whole, 
less  desirable  than  it  did  from  a  distance.  People  may  have  opposite 
orders  of  preference  for  gambles  when  asked  which  they  prefer  (which 
focuses  their  attention  on  how  likely  they  are  to  win)  and  when  asked 
how  much  they  would  pay  to  play  each  (which  highlights  the  amount  to 
win).  People  may  prefer  to  take  a  chance  at  losing  a  large  sum  of 
money  rather  than  absorb  a  small  sure  loss,  but  change  their  mind  when 
the  sure  loss  Is  called  an  Insurance  premium.  A  relatively  unimportant 
attribute  may  become  the  decisive  factor  In  choosing  between  a  set  of 
options  If  they  are  presented  In  such  a  way  that  that  attribute  affords 
the  easiest  comparison  between  them. 

Three  Important  features  of  these  shifting  judgments  are  (a)  people 
are  typically  unaware  of  the  potency  of  such  shifts  In  their  perspective, 
(b)  they  often  have  no  guidelines  as  to  which  perspective  Is  the  appro¬ 
priate  one,  and  (c)  even  when  there  are  guidelines,  people  may  not  want 
to  give  up  their  own  Inconsistency,  creating  an  Impasse. 


3.  LIMITS  TO  SETTING  LIMITS 


The  bottom  line  of  a  cost-benefit  analysis  Is  the  analyst's  best  guess 
at  the  relative  preponderance  of  costs  or  benefits.  Before  action  can 
be  taken,  one  must  know  how  good  that  best  guess  Is.  Depending  upon 
the  breadth  of  the  confidence  Intervals  on  the  "best-guess"  cost-benefit 
ratio,  one  might  want  to  collect  more  data.  Install  back-up  systems  to 
reduce  some  of  the  uncertainties,  or  abandon  the  project  for  one  whose 
consequences  are  better  known. 

The  analysts'  standard  practice  for  acknowledging  and  accommodating 
uncertainty  In  their  Inputs  Is  through  the  judicious  use  of  sensitivity 
analyses.  The  final  calculations  are  repeated,  each  time  using  an  alter¬ 
nate  value  of  one  troublesome  probability  or  utility.  If  each  reanalysis 
produces  similar  results,  then  the  case  Is  made  that  these  particular 
errors  do  not  matter.  One  way  of  viewing  the  research  on  judgnental 
biases  described  in  the  previous  section  is  that  it  merely  points  to 
additional  sources  of  error  calling  for  sensitivity  analysis. 

Unfortunately,  however,  there  are  no  firm  guidelines  as  to  which  Inputs 
might  be  In  error  or  what  Is  the  appropriate  range  of  possible  values 
to  be  tested.  The  possibility  of  judgnental  biases  would,  for  example, 
be  considered  only  If  the  analyst  were  aware  of  the  relevant  research 
and  took  It  seriously.  A  further  problem  with  sensitivity  analysis  Is 
that  It  typically  tells  us  little  about  how  the  uncertainty  from  different 
sources  of  error  Is  compounded,  or  about  what  happens  when  different 
Inputs  are  subject  to  a  comnon  bias.  The  untested  assunptlon  Is  that 
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errors  in  different  inputs  will  cancel  one  another  out,  rather  than 
compound  in  some  pernicious  way.7 

The  reasonableness  of  such  an  Independence  assumption  seems  weak  when  a 
set  of  judgnents  is  elicited  with  the  same  procedure,  inducing  the  same 
perspective.  For  example,  asking  about  preferences  In  a  mode  that 
Incorporated  a  reference  to  dollar  values  might  persistently  deflate 
the  expressed  Importance  of  environmental  or  other  less  tangible  values. 
To  take  an  example  from  the  elicitation  of  judgnents  of  fact,  the  U.S. 
Reactor  Safety  Study  called  upon  its  experts  to  assess  unknown  failure 
rates  by  the  "extreme  fractiles"  method,  choosing  one  nunber  so  extreme 
that  there  was  only  a  5X  chance  of  the  true  failure  rate  being  higher 
and  a  second  nunber  so  low  that  there  was  only  a  5X  chance  of  the  true 
rate  being  lower.  Research  conducted  with  a  variety  of  other  tasks  and 
subjects  have  shown  that  this  technique  routinely  produces  too  narrow 
confidence  intervals,  so  that  the  precision  of  these  estimates  Is  system¬ 
atically  exaggerated.8 

Even  if  sensitivity  analysis  could  handle  the  compounding  of  uncertainty, 
in  some  contexts  It  completely  misses  the  point.  Many  of  the  effects 
discussed  under  the  rubric  of  the  lability  of  values  refelct  the  Intro¬ 
duction  of  new,  possibly  foreign,  possibly  distorted  perspectives  into 
a  decision-making  process.  Invocation  of  sensitivity  analysis  will  not 


Flschoff,  B.  Decision  analysis:  Clinical  art  or  clinical  science? 

In  L.  Sjoberg  &  J.  Wise  (Eds.),  Proceedings  of  the  Sixth  Research  Con- 
ference  on  Subjective  Probability,  Utility  and  Decision-Making  (Warsaw) 
197? ,  In  press). 

^Lichtenstein,  S. ,  Flschoff,  B.  &  Phillips,  L.D.  Calibration  of  pro¬ 
babilities:  The  state  of  the  art.  In  H.  Jungermann  &  G.  deZeeuw  (Eds.), 
Decision  Making  and  Change  in  Hunan  Affairs  (Amsterdam:  D.  Reidel, 
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excuse  the  Imposition  of  an  ellcltor's  perspective  on  the  respondent. 

Nor  will  it  handle  shifts  in  perspective  that  lead  to  reversals  of 
preference. 

In  the  end,  determining  the  quality  of  an  analysis  is  a  matter  of  judgnent. 
Someone  must  intuit  which  Inputs  are  dubious  and  which  alternate  values 
should  be  Incorporated  in  sensitivity  analyses.  Essentially,  that  some¬ 
one  must  decide  how  good  his  or  her  own  best  judgment  Is.  Unfortunately* 
an  extensive  body  of  research  suggests  that  people  are  overconfident  in 
the  quality  of  their  own  judgnent.  Indeed,  people  have  been  found  to  be 
so  overconfident  in  their  degree  of  general  knowledge  that  they  will 
accept  highly  disadvantageous  bets  based  on  their  confidence  judgnents. 
Furthermore,  this  bias  seems  to  be  Impervious  to  instructions,  familiar¬ 
ity  with  the  task,  question  format  and  various  forms  of  exhortation 
9 

toward  modesty. 

A  particularly  relevant  version  of  this  overconfidence  emerges  when 
people  are  asked  to  judge  the  completeness  of  the  representation  of  a 
problem.  Research  here  has  shown  a  persistent  tendency  to  underestimate 
what  Is  left  out  and  overestimate  what  Is  known.  As  before,  while  most 
of  this  psychological  evidence  Is  derived  from  work  with  laypeople,  there 
Is  some  systematic  and  considerable  anecdotal  evidence  of  similar  process¬ 
es  at  work  with  experts.  Among  the  generic  kinds  of  Issues  whose  omission 
might  not  be  adequately  noted  by  energy  analysts  are:  (1)  the  imagina¬ 
tive  ways  In  which  hunan  error  can  mess  up  a  system;  (b)  the  range  and 
rate  of  possible  changes  In  people's  values  and  behavior  regarding  energy 
consunptlon,  (c)  the  nunber  of  unknown  or  undetected  physical,  biological. 


9Ib1d;  Flschoff,  B. ,  Slovic,  P.  &  Lichtenstein,  S.  Knowing  with  cer¬ 
tainty:  The  appropriateness  of  extreme  confidence.  Journal  of  Experi¬ 
mental  Psychology:  Hunan  Perception  and  Performance.  1977.  3,  552-564. 
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or  psychological  effects  of  a  new  system;  and  (d)  the  Interrelation 
between  system  components  (e.g. *  common  mode  failures  or  the  possibility 
of  a  system  falling  because  a  back-up  component  has  been  removed  for 
routine  maintenance).*® 

No  analysis,  performed  In  real  time  and  with  finite  resources,  claims 
to  be  complete  or  error  free.  Indeed,  all  responsible  analysts  Include 
sensitivity  analyses  with  their  reports.  The  preceding  discussion 
suggests,  however,  that  It  Is  hard  to  assess  the  adequacy  of  these 
analyses.  We  have  an  urgent  need  for  a  better  understanding  of  what 
errors  may  enter  Into  an  analysis,  how  virulent  they  are,  how  they  are 
propagated  and  compounded  throuqh  the  analysis,  what  can  be  done  to 
reduce  their  Impact,  how  we  can  assess  their  total  Impact,  and  what 
that  assessment  means  In  terms  of  action.  In  a  sense,  what  we  need  Is 
an  error  theory  for  cost-benefit  analysis,  supplemented  by  some 
empirical  study  of  the  fallibility  of  analyses  conducted  In  the  past. 

The  qualifications  accompanying  many  (or  most)  analyses  Include  reference 
to  what  could  have  been  done  with  greater  time  and  resources.  These  two 
coeanodltles  are,  however,  always  going  to  be  limited  and  we  must  know 
how  well  cost-benefit  analyst'  serves  us  under  realistic  constraints. 

One  conclusion  of  such  an  assessment  might  be  that  cost-benefit  analysis 
Is  useless  unless  XI  of  the  total  budget  can  be  Invested  In  It;  another 
might  be  that  virtually  all  the  value  of  a  cost-benefit  analysis  comes 
from  structuring  the  problem  and  conducting  a  few  back-of-the-envelope 
calculations  within  that  structure.  A  third  possible  conclusion  Is 


*  Flschoff,  B. ,  Slovlc,  P.  &  Lichtenstein,  S.  Fault  trees:  Sensitivity 
of  estimated  failure  probabilities  to  problem  representation.  Journal 
of  Experimental  Psychology:  Human  Perception  and  Performance,  T97B7"?, 
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that.  In  most  situations,  the  judgnental  components  of  cost-benefit 
analysis  are  so  essential  and  so  deeply  burled  that  conducting  a  formal 
analysis  merely  creates  an  aura  of  solvability  around  problems  that  are 
quite  dimly  understood. 
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4.  LIMITS  OF  SCOPE 


Like  all  other  procedures,  cost-benefit  analysis  deals  with  only  a  seg¬ 
ment  of  the  management  problem.  The  crucial  question  here  Is  whether 
that  segnent  can  stand  alone  and  Is  able  to  contribute  to  the  rest  of 
the  process  or  whether  Its  Internal  logic  disintegrates  when  confronted 
with  broader  realities. 

The  segnent  addressed  by  cost-benefit  analysis  Is  that  most  amenable  to 
formal  analysis  and  least  accessible  to  Individuals  without  technical 
expertise.  This  Is  most  certainly  true  in  variants  that  rely  heavily 
on  tools  like  shadow  pricing  or  revealed  preferences  to  deduce  what  the 
people  want  without  asking  them.  Some  variants,  like  decision  analysis, 
try  to  overcome  this  bias  by  Incorporating  elicitation  procedures  that 
can.  In  principle,  be  used  with  corporate  executives,  government  regula¬ 
tors  or  people  off  the  street.  Despite  such  efforts,  however,  the  very 
sophistication  and  centralization  of  the  analysis  gives  added  weight  to 
the  opinions  of  those  who  are  articulate  and  close  to  the  analyst. 

When  analytic  resources  are  limited,  the  analyst  must  take  cues  from 
someone  about  how  to  restrict  the  alternatives  and  consequences  consi¬ 
dered.  That  someone  Is  likely  to  be  the  one  who  cownlssioned  the  study. 
If  cownlssl oners  all  come  from  one  sector  of  society  and  consistently 
prefer  (or  reject  out  of  hand)  particular  kinds  of  solutions  or  conse¬ 
quences,  a  persistent  bias  may  be  produced.  Such  bias  would  determine 
what  Issues  are  never  analyzed  and  how  results  are  presented.  If  the 
cownlssl oners  are  public  officials,  there  may  be  a  strong  predisposition 
toward  reports  that  bury  uncertainties  and  delicate  assunptlons  In 
sophisticated  technical  machinations  or  In  masses  of  undigested  data. 
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If  one  examines  the  public  criticisms  to  which  cost-benefit  analyses  are 
subjected,  a  number  of  themes  emerge.  One  Is  that  there  Is  usually  a 
population  of  experts  who  are  angered  because  the  sensitivity  analysis 
did  not  Include  what  they  believe  to  be  appropriate  alternative  values 
for  some  Inputs.  When  these  experts  view  themselves  as  having  qualifi¬ 
cations  rivalling  those  of  those  experts  who  conducted  the  study,  the 
obvious  Implication  Is  that  even  the  elite  community  Involved  Is  somewhat 
restricted,  perhaps  for  reasons  of  political  or  academic  power. 

A  second  group  of  critics  views  the  segmentation  of  participants  as  a 
more  serious  Issue  than  the  substantive  topic  at  hand.  Their  main  con¬ 
cern  Is  that  the  use  of  analysis  transfers  power  for  societal  decision 
making  to  a  technical  elite.  In  effect  disenfranchising  the  lay  citizenry. 

One  might  argue  that  given  the  vagaries  of  lay  judgment  described  above, 
such  a  transfer  of  power  Is  In  the  best  Interests  of  even  that  lay 
public.  Let  someone  competent  do  the  job;  we’ll  all  be  better  off.  The 
counter- argument  has  several  facets.  One  Is  that  every  analysis  requires 
a  variety  of  judgments  that  might  just  as  well  be  performed  by  lay  people. 
Regarding  questions  of  fact,  when  they  are  forced  to  go  beyond  their 
tools  and  data  and  rely  on  Intuition,  experts  may  be  little  better  than 
non-experts.  Regarding  questions  of  value,  being  close  to  the  action 
should  not  confer  superiority  on  experts'  beliefs.  The  second  part  of 
the  argument  Is  that  there  are  higher  goals  than  maximizing  the  effi¬ 
ciency  of  a  particular  project.  These  Include  developing  an  Informed 
citizenry  and  preserving  democratic  Institutions.  The  process  may  be 
more  Important  than  the  product,  making  It  Important  to  devote  the 
resources  needed  to  make  meaningful  public  participation  possible. 

Such  participation  requires  new  tools  for  communicating  with  the  public, 
both  for  presenting  technical  Issues  to  lay  people  and  for  eliciting 
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values  from  them.  It  may  also  require  new  social  and  legal  forms,  such 
as  hiring  representative  citizens  to  participate  In  the  analytic  process, 
thereby  acquiring  the  expertise  needed  to  confer  the  Informed  consent  of 
the  governed  on  whatever  decision  Is  eventually  reached.  Such  a  format 
might  be  considered  a  science  court  with  a  lay  jury.  It  would  consider 
cost-benefit  analysis  as  one  Input  to  Its  proceedings.  It  would  also 
place  the  logic  of  jurisprudence  above  the  logic  of  economic  analysis, 
acknowledging  that  there  Is  no  formal  way  to  sunmarlze  the  Issues  at 
hand. 


A  third  group  of  critics  objects  to  the  segmentation  of  the  energy  pro¬ 
blem  from  the  broader  context  of  social  Issues.  The  critics  often  fight 
dirty  or  irrationally  (from  the  perspective  of  the  formal  analyst) 
because  they  view  the  cost-benefit  analysis  as  one  arena  in  which  politi¬ 
cal  struggles  are  waged.  Those  struggles  have  a  different  logic  than 
that  of  economic  analysis.  In  them.  It  may  be  fair  to  engage  In  uncon- 
structlve  criticism,  viciously  poking  holes  in  analyses  If  the  results 
do  not  support  one's  position.  It  may  even  be  legitimate  to  ridicule 
or  chastize  analysts  for  Ignoring  issues  (like  Income  distribution)  that 
were  outside  their  analytic  mandate. 

Some  representatives  of  this  position  would  argue  that  the  very  reason¬ 
ableness  of  formal  analysis  involves  a  political-ideological  assunption, 
namely,  that  society  Is  sufficiently  cohesive  and  conmon-goaled  that  Its 
problems  can  be  resolved  by  reason  and  without  struggle.  Although  such 
a  "get  on  with  business"  orientation  will  be  pleasing  to  many.  It  will 
not  satisfy  all.  For  those  who  do  not  believe  that  society  Is  In  a  fine 
tuning  stage,  a  technique  that  falls  to  mobilize  public  consciousness 
and  Involvement  has  little  to  recommend  It. 
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Thus,  there  are  logics  other  than  that  of  cost-benefit  analysis,  coming 
from  legal,  political,  even  revolutionary  theory.  Like  the  various  ways 
of  implementing  the  basic  cost-benefit  framework,  each  embodies  both 
ideological  predispositions  and  notions  of  how  society  operates.  Con¬ 
sidering  these  perspectives  and  the  impact  that  problem  formulation  can 
have  on  people's  judgnents  of  their  own  values  and  the  tendency  for  such 
analysis  to  create  an  aura  of  solvability,  cost-benefit  analysis  no 
longer  appears  as  a  value-neutral  procedure.  This  does  not  mean  that  it 
Is  not  or  cannot  be  made  into  the  one  technique  most  compatible  with  or 
capable  of  incorporating  the  broadest  range  of  values  in  a  particular 
society.  It  does  mean  that  a  political  position  of  sorts  is  being  taken 
when  one  adopts  the  procedure. 

Like  most  choices  Involving  ideologies,  questions  of  taste  are  somewhat 
disciplined  by  questions  of  reality.  Those  who  oppose  cost-benefit 
analysis  are  responding  in  part  at  least  to  social  concerns  and  facts  to 
which  the  analysis  is  relatively  or  totally  deaf.  Attention  to  these 
concerns  can  strengthen  an  analysis  and  heighten  its  impact.  For  example, 
an  analysis  that  ignores  question  of  equity  will  often  be  overturned  by 
those  who  come  out  on  the  short  end  of  the  project  in  question.  Rather 
than  let  their  work  become  a  mmber  game  with  no  real  effect,  many 
analysts  have  attempted  to  exploit  the  kernel  of  truth  in  their  critics' 
arguments  and  Incorporate  equity  considerations.  A  more  political  per¬ 
spective  might  also  help  one  realize  that  formal  analyses  deal  with  ideal 
types  often  having  no  representation  in  reality.  It  is  fairly  easy  to 
become  enamored  of  abstractions  and  analyze  projects  that  are  never  Imple¬ 
mented  in  the  way  or  at  the  time  they  are  proposed.*1  Although  the 


**Majone,  G.  The  feasibility  of  social  policies.  Policy  Sciences,  1975, 
6,  49-69. 
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method  may  be  tricky,  one  could  respond  to  this  challenge  by  considering 
ensembles  of  possible  representations  of  the  proposed  project  (l.e., 
ways  In  which  It  might  be  carried  out)  or  by  requiring  periodic  updates 
of  an  analysis  as  the  facts  change.  A  broader  perspective  could  motivate 
analysts  to  specify  the  assumptions  about  society  upon  which  their 
analyses  are  predicated  and  heighten  their  sensitivity  to  the  tenuousness 
of  those  assumptions.  In  the  extreme.  It  might  even  lead  them  to  reject 
analytical  mandates  that  separate  projects  from  their  social  context  In 
ways  that  are  not  meaningful. 


5.  CONCLUSION 


How  well  does  cost-benefit  analysis  fare  according  to  the  various  criteria 
listed  In  Table  1?  What  work  is  needed  to  make  it  fare  better?  Even 
though  the  preceding  discussion  has  focused  on  some  of  these  topics  at 
the  expense  of  others,  a  few  words  about  each  will  give  a  flavor  of 
what  a  fuller  consideration  might  reveal. 

5.1  Logically  Sound 

The  cost-benefit  family  of  analytic  procedures  has  a  logical  foundation 

that  is  both  carefully  thought  out  and  widely  accepted  in  both  political 

and  academic  circles.  As  a  result,  both  its  failings  and  its  assets  are 

better  documented  than  those  of  its  competitors.  Although  there  are 

still  important  technical  and  conceptual  problems  to  work  out  within  the 

12 

cost-benefit  framework,  a  useful  investment  of  energy  might  be  trying 
to  clarify  the  relationship  between  that  framework  and  the  logic  (or 
ostensible  logic)  of  other  approaches.  Are  they  really  incompatible? 

Can  cost-benefit  analysis  be  elaborated  to  Incorporate  the  elements  of 
truth  embodied  by  the  alternatives?  Perhaps  the  weakest  competition  Is 
provided  by  the  "logic"  uncovered  in  studies  of  people's  intuitive  deci¬ 
sion  making.  Yet,  even  here,  it  is  worth  asking  whether  there  Is  not  a 
method  in  people’s  apparent  madness.  Are  there  not  decision-making  cri¬ 
teria  overlooked  by  formal  analysis  yet  essential  for  hunan  welfare  or 
psychological  well  being? 


12 

Pearce,  O.W.  "Social  Cost  Benefit  Analysis  and  Nuclear  Futures."  In 
L.  Sjoberg,  T.  Tyszka  &  J.  Wise  (Eds.),  Decision  analysis  and  decision 
processes.  (Lund,  Sweden:  Doxa,  in  press! 
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5.2 


Implementability 


Like  other  computational  enterprises,  cost-benefit  analysis  rises  or 
falls  on  the  strength  of  its  inputs.  While  enormous  strides  have  been 
made  to  develop  a  cumulative  data  base  on  various  topics,  all  too  often 
the  analyst  is  forced  to  rely  on  intuitive  judgments.  Judgnents  of  fact 
tend  to  be  subject  to  persistent  biases  that  are  only  now  beginning  to 
be  understood.  Further  work  is  needed  here,  particularly  in  studying 
the  judgments  of  experts.  Judgnents  of  value  tend  to  be  highly  labile 
and  subject  to  complex  and  subtle  manipulation  by  the  questioning  pro¬ 
cedure  used.  Research  is  needed  to  produce  techniques  and  settings  that 
enable  respondents  to  elucidate  their  own  opinions. 

5.3  Politically  Acceptable 

In  contrast  to  the  political  objections  to  cost-benefit  analysis  raised 
earlier,  one  may  cite  a  number  of  fundamental  assets.  The  most  important 
of  these  is  the  explicit  expression  of  its  structure  and  inputs,  all  of 
which  are  in  principle  open  to  question  and  revision.  To  realize  this 
potential  advantage,  several  developments  are  needed.  One  is  procedures 
for  communicating  technical  issues  to  laypeople  (including  regulators 
and  legislators  without  a  technical  background)  so  that  they  can  offer 
reasoned  critiques.  A  second  need  is  to  develop  some  way  for  critics  to 
perform  their  own  sensitivity  analyses,  incorporating  their  own  alterna¬ 
tive  values  for  various  inputs.  Such  an  opportunity  might  produce  some 
surprising  results,  showing  the  conclusions  of  analyses  to  be  much  more 
(or  less)  robust  than  they  initially  appeared.  It  could  also  help  allay 
fears  that  these  conclusions  represent  the  result  of  ingenious  nunber- 
fudglng  by  the  analysts.  In  the  back  of  many  cynical  critics'  minds 
must  lurk  the  thought  that  the  experts  have  played  around  until  they 


found  a  constellation  of  values  that  looks  benign  but  produces  the 
recommendation  most  favorable  to  a  particular  point  of  view. 

5.4  Respects  Institutional  Constraints 

The  Initial  popularity  of  cost-benefit  analysis  probably  was  due  to  Its 
fitting  well  Into  the  way  In  which  business  was  done  In  various  seats  of 
government.  Its  continued  success  may  be  due  to  the  ability  of  pro¬ 
ponents  to  shape  legal  and  governmental  proceedings  to  accommodate  this 
tool  further.  Its  future  prospects  may  depend  on  the  successful  resolu¬ 
tion  of  several  persistent  problems.  One  Is  that  It  assumes  a  single 
decision  maker;  difficulties  arise  when  there  are  many  hands  Involved 
and  many  views  to  be  Incorporated.  A  second  problem  Is  that  It  Is  a 
one-time  analysis;  as  a  result.  It  Is  not  as  responsive  to  changing  con¬ 
tingencies,  preferences  and  scientific  data  as  are  the  bureaucracies  It 
Is  designed  to  serve.  A  third  problem  Is  that  It  requires  a  level  of 
analytical  expertise  not  possessed  by  many  of  the  Individuals  Involved 
In  Its  use,  producing  aberrations  and  frustrations. 

5.5  Open  to  Evaluation 

As  mentioned,  cost-benefit  analysis  claims  to  be.  If  nothing  else,  open 
to  Inspection.  To  realize  the  promise  of  this  claim,  several  develop¬ 
ments  are  needed.  Psychological  research  must  find  ways  to  help  people 
appraise  the  limits  of  their  own  knowledge  or,  falling  that,  ways  to 
assess  how  confident  they  should  be  given  how  confident  they  say  they  are. 
Theoretical  efforts  are  needed  so  that  analyses  produce  better  assessments 
of  their  own  limits  and  derive  the  action  Implications  of  that  cumulative 
uncertainty.  Since  such  efforts  have  their  own  Inherent  limits,  empirical 
work  Is  needed  to  review  past  analyses  to  explore  their  foibles  and  contri¬ 
bution  (or  lack  of  It)  to  the  management  process. 
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5.6 


Creates  No  Side  Effect 


Disenfranchising  the  lay  public  is  one  possible  side  effect  of  the  wide- 

scale  adoption  of  cost-benefit  analysis  about  which  some  segnents  of  the 

public  are  quite  agitated.  Denigrating  the  importance  of  consequences 

that  cannot  readily  be  expressed  in  dollar  or  other  quantitative  terms 

(e.g.,  extinction,  aesthetic  degradation)  Is  another.  Making  a  fetish 
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out  of  currently  enjoyed  benefits  is  a  third.  In  general,  though, 
the  implications  of  having  a  cost-benefit  society  are  poorly  understood. 

5.7  Promotes  Long-Term  Effective  Management 

Cost-benefit  analysis,  particularly  its  sensitivity  analysis  component, 
has  been  Instrumental  in  setting  the  research  agendas  of  those  concerned 
with  understanding  the  effects  of  technological  projects.  To  the  extent 
that  the  priorities  of  such  analyses  are  correct,  this  Is  a  major  con¬ 
tribution  to  creating  a  base  of  data  relevant  to  sound  management.  It 
has  also  provided  a  framework  within  which  talented  economists  could 
apply  themselves  to  these  problems.  Finally,  although  It  is  criticized 
for  emphasizing  product  over  process,  the  cost-benefit  analysis  frame¬ 
work  has  probably  raised  the  level  of  debate  In  many  settings  and 
broadened  the  understanding  of  even  Its  harshest  critics. 


There  is  no  verdict  on  cost-benefit  analysis  per  se.  One  must  consider 
It  In  the  light  of  alternative  approaches  and  In  the  context  of 


*^M1shan,  E.J.  What  Is  wrong  with  Roskl 117  In  R.  Layard  (Ed.),  Cost 
Benefit  Analysis.  (New  York:  Penguin,  1972) 


particular  situations  that  might  accentuate  Its  strengths  or  weaknesses. 
One  must  consider  not  only  cost-benefit  as  It  Is  today,  but  as  It  can 
be  Improved.  One  must  consider  not  only  the  nice  Idea  and  the  sparkling 
theory,  but  the  Integrity  with  which  It  will  be  applied. 

Given  the  limits  to  hunan  judgment  and  consensus  described  here.  It  Is 
unlikely  that  cost-benefit  analysis  In  a  pure  form  will  ever  be 
practiced  or  followed  anywhere.  The  critical  question  then  becomes, 
does  It  degrade  gracefully? 
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Research  has  shown,  however,  that  probability  judgments  are  often  quite 
unreliable  and  prone  to  systematic  biases,  while  judgments  of  value  are 
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of  the  procedures  to  address  critical  Issues  In  the  management  process  they 
are  designed  to  abet.  )Jhese  Issues  Include  the  acceptability  of  the  politi¬ 
cal  philosophy  underlying  such  procedures,  the  feasibility  of  Implementing 
their  recomnendations,  and  the  generation  of  managerial  options.  The  con¬ 
tribution  of  cost-benefit  analysis  may  be  enhanced  by  reducing  whichever 
of  these  problems  are  tractable,  acknowledging  those  that  are  not,  and  clari¬ 
fying  the  responsible  role  of  cost-benefit  analysis  In  the  management  pro¬ 
cess. 
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